The human brain contains around 1011 neurons and 1015 synapses. Neurons, synapses and the networks of them that form the human brain are very complex biological systems. FIG. 1A shows a simplified diagram of a biological neuron. In FIG. 1A, the neuron receives multiple excitatory input current signals (i1, i2, i3 . . . ) and produces a single output signal vout. There is a delay through the axon, which may be referred to as axonal delay. FIG. 1B shows an example of a typical output signal. It consists of a stream of spikes, which are pulses of short duration. The output information is encoded into the timing of these spikes (t1, t2 . . . ).
FIG. 1C shows a simplified model of the synapse circuit. The input terminal of the synapse is designated to receive the output voltage signal of a presynaptic neuron. This voltage is called the presynaptic input voltage and is denoted as vpre. The output terminal of the synapse is designated to provide a current into the input node of the postsynaptic neuron. This output current of the synapse is denoted as is.
Neural computers have been used to model the behavior of neurons and synapses and circuits for modeling their behavior have been proposed. In U.S. patent application Ser. No. 13/151,763, filed Jun. 12, 2011, J. Cruz-Albrecht, P. Petre and N. Srinivasa, describe a “High-Order Time Encoded Based Neuron Circuit”. The circuit described has many biological mechanisms but does not include the circuits to emulate the features of kinetic dynamics, homeostatic plasticity, and axonal delays.
Kinetic dynamics refer to the signal dynamics associated with the synapses of a neuron. In particular, kinetic dynamics refers to the time evolution of a synapse output response from a spike input. This time response has the shape of an exponential decay. Homeostatic plasticity refers to the capacity of neuron networks to regulate their own excitability relative to neural network activity. This self-regulation operates to evolve the output average spike rate over the long term to a target value. Axonal delays refer to delays in an axon, which typically conduct electrical impulses away from the neuron's cell body. The delay is associated with the time for a spike to be transmitted across an axon. An axon connects a neuron core producing a spike to target synapses that receive delayed versions of that spike.
In U.S. Pat. No. 7,822,698, issued Oct. 26, 2010, J. Cruz-Albrecht and P. Petre describe “Spike Domain and Pulse Domain Non-Linear Processors”. The neuron circuits described in U.S. Pat. No. 7,822,698 have a spike domain feature but do not include the circuits to emulate features of kinetic dynamics, homeostatic plasticity, and axonal delays.
J. Cruz-Albrecht, M. Yung and Srinivasa describe another circuit in “Energy-Efficient Neuron, Synapse and STDP Circuits,” IEEE Trans. on Biomedical Circuits and Systems, pp. 246-256, Vol. 6, No. 3, June 2012. This circuit does describe a neuron core but also does not include any circuitry to provide features for kinetic dynamics, homeostatic plasticity, and axonal delays.
J. Lazzaro describes yet another circuit in “Low-Power Silicon Spiking Neurons and Axons,” IEEE Symposium on Circuits and Systems, pp. 2220-2223, 1992. This paper describes a circuit for homeostatic plasticity and kinetic dynamics. However a capacitor is required for each input to a synapse associated with a neuron, which can be a very large number of capacitors.
C. Bartolozzi et al. in “Silicon Synaptic Homoestasis” Brain Inspired Cognitive Systems, Oct 2006 describe a circuit with a type of axonal delay. But the circuit requires two capacitors for each delay stage.
J. Schroyer in U.S. Pat. No. 3,569,842 issued Mar. 9, 1971 and titled “Pulse Delay Circuit” describes a pulse delay circuit that supports a delay longer than the pulse width; however, the circuit does not preserve the pulse width information. The output pulse width is instead pre-set to a fixed value as a function of the circuit parameters.
J. Wharton in U.S. Pat. No. 3,824,411 issued Jul. 16, 1974 and titled “Pulse Delay Circuit” describes a circuit that delays the rising and falling edges of the pulse independently, hence preserving the pulse width information; however, the circuit does not support a greater delay than the width of the pulse.
Digital pulse delay circuits, while providing very flexible delays and pulse width control, suffer from high complexity and circuit area requirements, and because of the large number of neurons in a neural net, digital pulse delay circuits are cumbersome.
In many applications such as in a digital system, the pulse width is often known or pre-defined rather than field dependent. The delay circuit described by J. Schroyer in U.S. Pat. No. 3,569,842 does not preserve the input pulse information but rather generates a fixed pulse width for its output. In a neural circuit, the pulse or spike width varies and does have an impact on the response of the circuit receiving it. Also, in a digital system, long delays or multiple delays are usually implemented by cascading many stages of delay circuit where the delay of each is less than the pulse width, such as described by J. Wharton in U.S. Pat. No. 3,824,411.
What is needed is a circuit that overcomes the disadvantages of the prior art. It would be desirable to have a better more compact delay circuit. It would also be desirable to reduce the complexity of the circuitry due to the challenge of modeling the human brain, while more accurately modeling the biological properties of neurons and synapses. The embodiments of the present disclosure answer these and other needs.